1. Field of the Invention
This invention relates to the field of software system building, and, more specifically, to system building using an interpreter.
2. Background
Often in the development of a large application, the construction of software code for implementing the application is divided into separate code construction efforts centered around individual code objects or components having their own algorithms and data. System building is applied to form the separate code components into the finished software object or objects, referred to as "derived objects," which embody the application.
In the prior art, the construction, or "building," of a software application or system, i.e., the reduction of coded objects or components and their respective linking rules into libraries, executable source code, or other derived objects, is performed in a single complicated, pass-fail build operation. Either the system is completely compiled during the build operation, or the build operation must be restarted from the beginning. No safe mechanism exists that allows for an interrupted build operation to proceed from any point within the operation other than the beginning.
System Building
The construction of derived objects, such as program executables, referred to as "building," entails the combination of a set of input objects and a set of rules that describe how to manipulate the input objects to produce the derived objects. These rules may consist of scripted instructions for linking objects, constructing interfaces and establishing system inputs otherwise not provided by the objects. The construction of the scripted rules may be generated in a text file by a developer using a text editor to generate the rules in a scripting language readable by a system builder process. Also, the rules may be generated as an intermediate output file from a system development tool.
Some system development tools provide graphic user interfaces (GUIs) for the developer to identify objects from a library for use in the system and to graphically represent the desired object links, interfaces, and inputs for a selected set of identified input objects. The system development tool composes an intermediate output file representing the necessary rules, e.g., in a script format, for supporting the system that was defined by the developer using the tool's GUI. The format of the script is designed to be interpreted by the system builder process to facilitate construction of the desired derived objects.
The scripted rules are provided as input to the system builder process which constructs the system, e.g., by compiling the object code for the specified input objects and linking the code in the appropriate manner. The system is evaluated by the system builder process, and derived objects are generated as output. The derived objects are usually provided in the form of an executable file, though intermediate representations of the derived objects are also possible.
Two examples of building environments are the "make" system provided in the UNIX environment, and the "Vesta" system developed by the Systems Research Center of Digital Equipment Corporation. The "make" system is designed with mutable objects in mind, relying on a dependency structure. However, the "Vesta" system is designed for attaining repeatability (i.e., each build is the same as every other build for a given set of input objects). In Vesta, repeatability is achieved using immutable objects. Immutable objects are not susceptible to changes. However, mutable objects are prone to changes such as modifications to code or data within the object. Typically, modified objects can be identified from the change in the modification date assigned to the file that contains the modified code or data.
In the "make" system, a developer creates a "makefile" which describes the software system in terms of the source and object modules that go into the system. Dependencies for each activity in the "makefile" are explicitly listed, such that when a first object is dependent on a second object and the second object has changed, the first object is rebuilt. This enables changes, such as module version updates, to propagate between objects. However, "make" relies on file dates to determine what to rebuild. Whether or not an object is rebuilt depends upon the time stamps for the object and its constituents. This approach is sufficient only if progress is made along the timeline in a positive direction. If an earlier version of a first object is determined to be superior to a later version of the same object, and the earlier version is reinstated in the later version's place, the timestamp of the first object will reflect the file date of the earlier version. "Make" will not rebuild any objects dependent on the first object because the time stamp of the first object predates the timestamp of the dependent object. Further, the "make" system does not provide repeatability. Because objects are permitted to change, successive builds can yield differing results.
The Vesta system uses an input "model file" similar to a "makefile." However, the Vesta system does not use the same dependency format. Vesta guarantees repeatability in system building by using immutable objects. The input objects for a particular system build are not permitted to change, and since the system build has no other external inputs, repeatability is achieved. The immutability of the input objects is maintained by placing the input objects in a project area in long term storage assigned to a specific system build operation. No objects within the project area are permitted to be modified. Updated copies of input objects are placed in a new project area, and are considered to be a part of a different system build operation.
Both the "make" system and the Vesta system use pass-fail build operations wherein an interruption of the system build necessitates rebuilding from the beginning. The Vesta system implements a caching system to recover some utility from previous builds. In the caching system, certain operations in the evaluation of the input "model file" are stored in the Vesta cache. When the same operation is called for in a subsequent build, Vesta searches the cache for the same operation and uses the cached evaluation parameters rather than recomputing the operation. Thus, evaluation of the "model file" may experience a speed increase for some operations that were previously evaluated. However, implementation of the caching function can result in a speed reduction for all operations as the cache is queried. Further, rebuilding or re-evaluation must still proceed through the "model file" from the beginning.
FIG. 1 is a flow diagram of a general system building process. In step 100, the developer determines which objects will be used as input objects for the system. These objects may comprise standard objects provided from a third party, objects the developer has constructed himself, or objects provided by a third party and subsequently modified by the developer to meet his needs. Once the set of input objects is determined, in step 101, an object configuration file is constructed to define how the system is to be built from the selected set of input objects. The object configuration file contains references to the input objects, and typically specifies the steps to be taken to compile the input objects and link them together in the desired manner to generate the desired derived object or objects. As earlier described, the object configuration file is typically a high level script file generated with a text editor or with a specialized system development GUI application.
In step 102, the object configuration file is provided as input to a system builder application which evaluates the object configuration file and performs the scripted operations to generate the derived objects. For large systems having many input objects, the system builder application may have many time-consuming tasks to perform. For instance, the system builder application compiles each input object individually, e.g., by making a call to a separate compiler process and waiting for the compiler to return a completion indication. Therefore, the building operation of step 102 can take several hours or more to complete.
In step 103, a branching occurs based on whether the building operation of step 102 finishes in its entirety, or whether the building operation is halted some time prior to completion. If the build operation halts before completion, e.g., due to a system crash, an error in the object configuration file, etc., then the build operation of step 102 must be restarted from the beginning. If the build operation completes successfully, then the generated derived objects are implemented in step 104.
FIG. 2 is a general block diagram of a system building process. Standard objects 200 and user objects 201 are shown as inputs into system development tool 205. The output of system development tool is object configuration file 202, which comprises references to the input objects and rules for linking the input objects together, herein represented by block 203. Object configuration file 202 is provided to system builder 206 for evaluation. Derived objects 207, which may comprise one or several derived objects, are generated by system builder 206 during the evaluation process. System development tool 205 and system builder 206 are components of system building tools 204.
Standard objects 200 comprise objects provided by a third party and selected as input objects for the building operation. These objects include standard code components such as "C" code standard libraries and other general code components used to support the user objects 201. User objects 201 are input objects which the developer may have coded himself, or which the developer may have received from another source and optimized for use in the system. The input objects represented by standard objects 200 and user objects 201 are the building blocks of the desired system. In the Vesta system, these input objects are typically kept in a designated project directory in long term storage to maintain their immutability. I.e., the input objects are set aside in long term storage to prevent unintentional modification of the object files.
Once the developer has in mind which objects are to be input objects for the system, the developer uses system development tool 205 to generate the rules or steps for building the system. As described above, system development tool 205 may be a GUI tool for selecting objects (e.g., by file name) and assigning links, or system development tool 205 may be a standard text editor wherein the developer generates a text file containing the scripted rules for building the system. The scripted rules generated by the system development tool are stored as object configuration file 202, which often has the appearance of a high-level language program.
For example, in the Vesta system, a fragment of an object configuration file, or "model file," has the following format:
______________________________________ DIRECTORY Main.mod = /vesta/project.a/object.1, Subs.mod = /vesta/project.a/object.2, IN { build = FUNCTION Prog, Compile IN Prog( (Compile(Main.mod), Compile(Subs.mod)) ); .cndot. .cndot. etc. ______________________________________
In the above Vesta model file, the "directory" portion of the fragment serves to provide unique identifiers for the input objects, "object.1" and "object.2" (e.g., with a reference to their directory location, i.e., within subdirectory "project.a" in the main "vesta" directory), and to bind the names, "Main.mod" and "Subs.mod," respectively, to the input object identifiers. The "Compile" and "Prog" functions are used to invoke a compiler and linker, respectively. When evaluated, the compound build statement effectively invokes the compiler twice, combines the results in a two-element list, and passes the list to the linker.
The Vesta "model file" format is recognizable as similar to that of a program written in a high-level language. The actual format of the object configuration file typically depends on the format expected by the particular system builder responsible for evaluating it. An actual object configuration file may comprise many statements and functions, and may be used to generate system executables, manual "MAN" pages, and other derived objects in the same build operation.
System builder 206 of FIG. 2 evaluates object configuration file 202, carrying out the instructions specified in file 202 to generate derived objects 207. In addition to evaluating file 202 to build the derived objects, the system builder may also apply type checking, debugging, and optimizing functions during the evaluation to identify improper build instructions in the object configuration file or to optimize generation of the derived objects.
Software Interpreters
Evaluation of the object configuration file is sometimes performed via an interpreter. An interpreter is a software mechanism that is used to execute a high-level program. The interpreter reads a high-level input program, in this case, the object configuration file, and causes the computer system to carry out operations consistent with those called for in the input program. In some instances, such as when a bytecode interpreter is used, the high-level input program is precompiled into bytecodes for evaluation by the interpreter. The interpreter typically runs on top of the operating system, but may also be designed to execute without an underlying operating system.
FIG. 3 is a block diagram of a computer system having an interpreter-based system builder. The foundation of the system is the computer machine hardware 303 comprising random access memory and a central processing unit having an architecture supporting a particular microcode instruction set. Operating system 302 resides in memory and is run by the central processing unit in machine hardware 303. Operating system 302 provides system-level support such as control over I/O functions, multi-processing, etc., and provides an environment for application-level programs.
Interpreter 301 and compilers/linkers 304 are software processes running on top of operating system 302. Object configuration file 202 is an input file to interpreter 301. Pursuant to the instructions within input file 202, the interpreter may perform some operation on the input object files 300, including directing the compilers/linkers 304 to compile and/or link specific input object files in a prescribed manner. The compiled and linked output file or files are stored in long term storage as derived objects 207.
A typical hardware machine, such as that represented by machine hardware 303, has a stack of microinstructions for carrying out a process, be it an application program or part of the operating system. The microinstructions designate particular operations to be performed on the contents of registers within the machine, such as "add the contents of registers A and B, and store the result in register A," or "load the contents at memory address X into register A." A register referred to as the "PC register" maintains a pointer to the next microinstruction in the stack. When a microinstruction has been carried out, the machine advances to the next microinstruction indicated by the PC register. The complexity of the individual microinstructions are determined by the specific microinstruction set supported by the machine architecture.
Interpreter 301 is designed to emulate the operation of a machine in its evaluation of input instructions. For this reason, the interpreter is sometimes referred to as a "virtual machine." The virtual machine provides a level of abstraction from the real machine carrying out the low level operations, i.e., hardware machine 303. Whereas the microinstruction set of machine 303 is fixed, the complexity of the input instruction set for the virtual machine is substantially arbitrary, so long as each instruction in the input instruction set of the virtual machine is realizable by some combination of microinstructions from hardware machine 303.
For example, the functions available to a standard microprocessor are often determined by an internal arithmetic logic unit (ALU) which may provide such register-level operations as simple addition, subtraction, bit-shifting, etc. Similarly, a virtual machine such as that provided by components 301-304 in FIG. 3, could provide higher level instructions such as "compile file A and store result as file B," or "link file A and file B, and store result as file C." In this example, files A, B and C comprise virtual structures similar to the registers of a real machine. The higher level instructions provided by the virtual machine are supported by software methods within the interpreter process and by linked processes, such as compilers/linkers 304, programmed to perform higher level operations. Different compiler processes may be provided for compiling objects in different computer languages.
Using an interpreter to evaluate an input program can be much slower than execution of a program that has been compiled into hardware machine readable code. This is due to the cost inherent in the emulation of a virtual machine. It would therefore be advantageous to be able to preserve the work performed by an interpreter in the event that the interpreter is interrupted. The time and resource consuming act of re-evaluating operations in an input file would be obviated. However, in general, no mechanism exists for preserving the forward progress of an interpreter.